Imaging an object using projected electromagnetic radiation and a rolling shutter

ABSTRACT

According to various, but not necessarily all, embodiments there is provided an apparatus comprising means for: capturing or causing capture of an image of an object using a rolling shutter having an aperture width and shutter scan speed; during the image capture, projecting or causing projection of electromagnetic radiation with a fixed-spatial, time-variable distribution onto the object, wherein the time-variable distribution of the projected electromagnetic radiation, the aperture width of the rolling shutter, and the shutter scan speed of the rolling shutter, are such that adjustment of one or more of these would not decrease a proportion of projected electromagnetic radiation captured which is directly reflected from a surface of the object.

TECHNOLOGICAL FIELD

Embodiments of the present disclosure relate to imaging an object usingprojected electromagnetic radiation and a rolling shutter. Someembodiments relate to imaging an object using projected electromagneticradiation and a rolling shutter to enable information about thesubsurface structure of the object to be obtained.

BACKGROUND

Information about the subsurface structure of an object can be obtainedfrom electromagnetic radiation that has undergone subsurface scattering.However, when electromagnetic radiation is projected onto the object,the majority of it is directly reflected from the surface of the object.The captured signal is therefore dominated by directly reflectedelectromagnetic radiation and it is accordingly challenging to obtaininformation about the subsurface structure from the captured signal.

BRIEF SUMMARY

According to various, but not necessarily all, embodiments there isprovided an apparatus comprising means for: capturing an image of anobject using a rolling shutter having an aperture width and shutter scanspeed; and, during the image capture, projecting electromagneticradiation with a fixed-spatial, time-variable distribution onto theobject. The time-variable distribution of the projected electromagneticradiation, the aperture width of the rolling shutter, and the shutterscan speed of the rolling shutter, are such that adjustment of one ormore of these would not decrease a proportion of projectedelectromagnetic radiation captured which is directly reflected from asurface of the object.

According to various, but not necessarily all, embodiments there isprovided a method comprising: capturing an image of an object using arolling shutter having an aperture width and shutter scan speed; and,during the image capture, projecting electromagnetic radiation with afixed-spatial, time-variable distribution onto the object. Thetime-variable distribution of the projected electromagnetic radiation,the aperture width of the rolling shutter, and the shutter scan speed ofthe rolling shutter, are such that adjustment of one or more of thesewould not decrease a proportion of projected electromagnetic radiationcaptured which is directly reflected from a surface of the object.

According to various, but not necessarily all, embodiments there isprovided a computer program that, when run on a computer, performs:causing capture of an image of an object using a rolling shutter havingan aperture width and shutter scan speed; and, during the image capture,causing projection of electromagnetic radiation with a fixed-spatial,time-variable distribution onto the object. The time-variabledistribution of the projected electromagnetic radiation, the aperturewidth of the rolling shutter, and the shutter scan speed of the rollingshutter, are such that adjustment of one or more of these would notdecrease a proportion of projected electromagnetic radiation capturedwhich is directly reflected from a surface of the object.

The following portion of this “Brief Summary” section, describes variousfeatures that may be features of any of the embodiments described in theforegoing portion of the ‘Brief Summary’ section. The description of afunction should additionally be considered to also disclose any meanssuitable for performing that function.

At a given aperture position of the rolling shutter, electromagneticradiation which is projected onto an area of the object outside a fieldof view from the aperture position and which has which has undergonesubsurface scattering into the field of view may be captured.

The time-variable distribution of the projected electromagneticradiation, the aperture width of the rolling shutter, and the shutterscan speed of the rolling shutter, may be set such that adjustment ofone or more of these would not decrease the proportion of projectedelectromagnetic radiation captured which is directly reflected from asurface of the object.

The time-variable distribution and the shutter scan speed may be suchthat over periods of the time-variable distribution in which theelectromagnetic radiation is not projected the change in the apertureposition is less than or equal to twice a target lateral distance forscattering.

The time-variable distribution of the projected electromagneticradiation may comprise a temporally-alternating distribution.

The time-variable distribution of the projected electromagneticradiation may comprise strobing.

The strobing may comprise an on-cycle of equal duration with anoff-cycle.

The time-variable distribution of the electromagnetic radiation maycomprise a temporal offset from the beginning of the image capture.

The fixed-spatial distribution of the projected electromagneticradiation may comprise a repeating pattern

The time-variable distribution of the projected electromagneticradiation, the aperture width of the rolling shutter, and the shutterscan speed of the rolling shutter may be controlled to: cause capture offirst strips of the object when second strips, adjacent the first, areilluminated by denser regions of the repeating pattern; and causecapture of the second strips while the projection of the electromagneticradiation is turned off.

The fixed-spatial distribution of the projected electromagneticradiation may comprise a fixed distribution of dots.

The projected electromagnetic radiation may comprise one or moreinfrared frequencies.

The shutter scan speed of the rolling shutter may be constant during theimage capture.

The aperture width of the rolling shutter may be constant during theimage capture.

The time-variable distribution of the projected electromagneticradiation, the aperture width of the rolling shutter, and the shutterscan speed of the rolling shutter, may be such that adjustment of one ormore of these would not decrease the proportion of projectedelectromagnetic radiation captured which is directly reflected from oneor more surfaces of one or more parts of the object which are ofinterest.

Multiple spatially-offset images of the object may be captured as arelative position of the apparatus and object varies.

The multiple spatially-offset images may be captured during jitter.

Two or more spatially-offset images may be fused into a combined imageof the object which comprises less of the projected electromagneticradiation directly reflected from a surface of the object than either ofthe two or more spatially-offset images.

Subsurface structure information obtained from the multiplespatially-offset images may be used to align the multiplespatially-offset images.

Multiple images of the object may be captured and, from one imagecapture to another, one or more of: a frequency of the projectedelectromagnetic radiation; an amplitude of the projected electromagneticradiation; the time-variable distribution of the projectedelectromagnetic radiation; the aperture width of the rolling shutter; orthe shutter scan speed of the rolling shutter, may be varied.

Surface properties of the object may be obtained. Based on the surfaceproperties and a target depth of scattering and/or target lateraldistance of scattering, a target frequency and/or amplitude for theprojected electromagnetic radiation may be determined.

The target depth may be based on the type of object to be imaged.

The target lateral distance of scattering may be based on a distributionof illumination of the object by the projected electromagneticradiation.

Determining the target frequency and/or amplitude for the projectedelectromagnetic radiation may be further based on a distance between theapparatus and the object and/or ambient lighting conditions.

Determining the target frequency and/or amplitude for the projectedelectromagnetic radiation may be further based on subsurface scatteringproperties of a material from which the object is composed.

The target frequency and/or amplitude may be differentially determinedin respect of different parts of the object.

The frequency and/or amplitude of the projected electromagneticradiation may be varied during the image capture such that differentparts of the object are illuminated with electromagnetic radiationprojected at respective target frequencies and/or amplitudes.

Guidance may be provided to a user which indicates a distance betweenthe apparatus and the object to facilitate the target depth ofscattering and/or target lateral distance of scattering.

The image of the object or the combined image of the object may beprocessed to filter directly reflected electromagnetic radiation fromthat which has undergone lateral subsurface scattering based on adistribution of illumination of the object.

The image of the object or the combined image of the object may beanalysed to determine subsurface structure information.

The object imaged may be a body part of a user. A map of the user'ssubsurface blood vessels may be obtained from the image of the object orthe combined image of the object. The user may be authenticated based onthe obtained map.

According to various, but not necessarily all, embodiments there isprovided examples as claimed in the appended claims.

According to various, but not necessarily all, embodiments there isprovided a method comprising: capturing or causing capture of an imageof an object using a rolling shutter; during the image capture,projecting or causing projection of electromagnetic radiation with afixed-spatial distribution comprising a repeating pattern onto theobject; and controlling timing of the rolling shutter and projection ofthe electromagnetic radiation to cause capture of first strips of theobject when second strips, adjacent the first, are illuminated by denserregions of the repeating pattern, and to cause capture of the secondstrips while the projection of the electromagnetic radiation is turnedoff.

According to various, but not necessarily all, embodiments there isprovided an apparatus comprising means for performing the method.

According to various, but not necessarily all, embodiments there isprovided a computer program that, when run on a computer, performs themethod.

According to various, but not necessarily all, embodiments there isprovided a method comprising: capturing or causing capture of multiplespatially-offset images of an object as a relative position of theapparatus and object varies due to jitter; during the image captures,projecting or causing projection of electromagnetic radiation onto theobject; and fusing or causing fusion of two or more of thespatially-offset images into a combined image of the object whichcomprises less of the projected electromagnetic radiation directlyreflected from a surface of the object than either of the two or morespatially-offset images.

According to various, but not necessarily all, embodiments there isprovided an apparatus comprising means for performing the method.

According to various, but not necessarily all, embodiments there isprovided a computer program that, when run on a computer, performs themethod.

BRIEF DESCRIPTION

Some examples will now be described with reference to the accompanyingdrawings in which:

FIG. 1 shows an example apparatus;

FIG. 2 shows an example method;

FIG. 3 shows an illustrative example;

FIGS. 4A-C show another illustrative example;

FIGS. 5A-C show another illustrative example;

FIGS. 6A-C show another illustrative example;

FIGS. 7A-C show another illustrative example;

FIG. 8 shows another illustrative example;

FIGS. 9A-C show another illustrative example;

FIGS. 10A-C show another illustrative example;

FIGS. 11A-C show another illustrative example;

FIGS. 12A-C show another illustrative example;

FIG. 13 shows another example method;

FIG. 14A-B show another illustrative example;

FIG. 15A-B show another illustrative example;

FIG. 16 shows another example method;

FIG. 17 shows another example method; and

FIG. 18 shows an example controller.

DETAILED DESCRIPTION

FIG. 1 shows an example of an apparatus 101 comprising a camera 103 orany other suitable means for capturing an image of an object 113 and aprojector 109 or any other suitable means for projecting electromagneticradiation 111 onto the object 113.

The apparatus 101 may be or be comprised within a user device, such as amobile phone or another portable computing device. The apparatus 101 mayalso comprise the controller 1801 illustrated in FIG. 18.

The camera 103 is operable in a rolling shutter mode. In the rollingshutter mode, an aperture 105 having a width which yields a field ofview (FOV) 107 less than that of the camera 103 as a whole is scannedrelative to the scene and image sensor(s). The rolling shutter mode maybe implemented mechanically or electronically.

The projector 109 projects electromagnetic radiation 111 infixed-spatial distribution. That is, the distribution in which theelectromagnetic radiation 111 is projected is spatially invariant.Whenever the electromagnetic radiation 111 is projected, it is projectedin the same spatial distribution.

In some, but not necessarily all, examples the fixed-spatialdistribution of the projected electromagnetic radiation is a fixeddistribution of dots. The projector 109 can be a dot projector.Micro-lenses can be used to simultaneously project the distribution ofdots. The fixed-spatial distribution may take the form of otherstructured light. The projected electromagnetic radiation 111 maycomprise one or more infrared frequencies. If so, the camera 103comprises infrared sensors. The projected electromagnetic radiation 111may comprise visible light. The projector 109 can be a transmitter in aLIDAR system.

The object 113 has a surface 115. A proportion of the projectedelectromagnetic radiation 111 will be directly reflected from thesurface 115. The object also comprises a subsurface region 117comprising subsurface structures 119 which may be of interest. Aproportion of the projected electromagnetic radiation 111 will bescattered by subsurface structure 119 of the object 113. In this figure,the directly reflected electromagnetic radiation is referenced as 121and the electromagnetic radiation which has undergone subsurfacescattering is referenced as 123.

At a given aperture position of the rolling shutter, electromagneticradiation 111 which is projected onto an area of the object 113 outsidea field of view 107 from the aperture position can be captured when ithas been scattered by subsurface structure 119 of the object 113 intothe field of view 107. That is, electromagnetic radiation 111 which isprojected onto an area of the object 113 outside a field of view 107from the aperture position and which has undergone subsurface scatteringinto the field of view 107 can be captured.

Capture of the electromagnetic radiation 123 which has undergonesubsurface scattering enables information about the subsurface structure119 of the object 113 to be extracted, provided that the electromagneticradiation 123 which has undergone subsurface scattering can bedistinguished from the directly reflected electromagnetic radiation 121.

The apparatus 101 enables the capture of an image from which informationabout the subsurface structure 119 of the object 113 can be extracted.

FIG. 2 shows an example of a method 201 of operating the apparatus 101to capture an image 209 in which information regarding the subsurfacestructure 119 of the imaged object 113 can be obtained. The method 201enables such an image 209 to be obtained even when using a projector 109which cannot vary the spatial distribution with which theelectromagnetic radiation 111 is projected.

The method 201 comprises, at block 203, beginning the capture of animage 209 of the object 113 using a rolling shutter. The image capturebegins at an initial aperture position and proceeds to multiple aperturepositions, progressing at a rate dictated by the shutter scan speed,until reaching a final aperture position. The image sampling rate of thecamera 103 may be synchronized with the aperture width and the shutterscan speed so as to capture the image 209 as a sequence ofnon-overlapping strips. These strips may each span fully a firstdimension of the image 209, and collectively span a second dimension ofthe image 209.

During the image capture, according to block 205 of the method 201,electromagnetic radiation 111 is discontinuously projected. During theimage capture the electromagnetic radiation 111 is at times projectedand at other times not projected. It is projected with the fixed-spatialdistribution, and in a time-variable distribution, onto the object 113.That is, when the electromagnetic radiation 111 is projected during theimage capture, it is projected in the same spatial distribution butwhether it is projected or not at a given time varies with time duringthe image capture. Accordingly, some of the strips which collectivelymake up the image 209 may have been captured while the electromagneticradiation 111 was projected onto the object 113 and some may have beencaptured while the electromagnetic radiation 111 was not projected.

In some examples the time-variable distribution of the projection of theelectromagnetic radiation 111 comprises a temporally-alternatingdistribution such as, for example, strobing. The temporally-alternatingdistribution comprises a repeating on-cycle, during which theelectromagnetic radiation 111 is projected, and a repeating off-cycle,during which the electromagnetic radiation 11 is not projected. Theon-cycle may be of equal duration with the off-cycle as illustrated inFIG. 4B or may be of different duration to the off-cycle as illustratedin FIG. 7B.

In some examples the time-variable distribution comprises a temporaloffset from the beginning of the image capture. For example, the imagecapture may begin and proceed for a time before the electromagneticradiation 111 is strobed.

At block 207 of the method 201 the image capture ends and the image 209of the object 113 is obtained. The image capture ends once a strip hasbeen captured at the final aperture position.

In the method 201 settings of camera-related parameters, such as theaperture width and shutter scan speed, and projector-related parameters,such as the time-variable distribution, are such that adjustment of oneor more of these would not decrease the proportion of projectedelectromagnetic radiation 111 captured which is directly reflected fromthe surface 115 of the object 113 (rays 121 illustrated in FIG. 1).Examples of adjustments and the effect on the proportion ofelectromagnetic radiation 111 captured which is directly reflectedelectromagnetic radiation 121 are shown in FIGS. 4A-C, 5A-C, 6A-C, 7A-C,9A-C, 10A-C, 11A-C, and 12A-C.

The aforementioned adjustments are those which are realizable in view ofconstraints on the camera- and projector-related parameters. Constraintscan include, for example: lower, and optionally upper, limits to theaperture width; upper, and optionally lower, limits to the shutter scanspeed; rise and decay times for the projector 109; and, where relevant,maximum strobing frequency.

By operating the apparatus 101 during the image capture with theaforementioned camera- and projector-related parameters, more often thannot during the image capture, the regions (strips) of the object 113which are captured while the electromagnetic radiation 111 is projectedare adjacent regions of the object 113 which are more denselyilluminated by the projected electromagnetic radiation 111. Accordingly,the amount of electromagnetic radiation 111 which has undergonesubsurface scattering into these regions increases relative to theamount which is directly reflected from the surface 115.

In some examples, the settings of the aforementioned parameters aresubject to additional constraints. For example, the time-variabledistribution and the shutter scan speed may be such that over periods ofthe time-variable distribution in which the electromagnetic radiation111 is not projected (e.g., during off-cycle of a temporally-alternatingdistribution) the change in the aperture position is less than or equalto twice a target lateral distance for scattering. The target lateraldistance for scattering is the lateral subsurface scattering distancethat parameters of the projected electromagnetic radiation 111 such asthe frequency and amplitude are controlled to achieve. As a result,projected electromagnetic radiation 111 incident on any part of theimaged region of the object 113 can be scattered by the subsurfacestructure 119 into the FOV 107 of the rolling shutter. Therefore, it ispossible to obtain information about the subsurface structure 119 acrossthe full extent of the imaged region of the object 113.

The time-variable distribution, aperture width and shutter scan speedmay be fixed, selected from predetermined settings, or dynamicallyupdated.

In examples in which the settings of the parameters are fixed, forexample by the manufacturer, they may be fixed such that the proportionof the projected electromagnetic radiation 111 captured which isdirectly reflected from the surface 115 of the object 113 can beminimized when the apparatus 101 is positioned a specified distance awayfrom the object 113. The user of the apparatus 101 may be instructed orguided to position the apparatus 101 at the specified distance from theobject 113.

In other examples the method 201 comprises, prior to block 203, settingthe time-variable distribution of the projected electromagneticradiation 111, the aperture width of the rolling shutter and the shutterscan speed of the rolling shutter such that adjustment of one or more ofthese would not decrease the proportion of projected electromagneticradiation 111 captured which is directly reflected from the surface 115of the object 113. The settings can be determined from a look-up table.The look-up table may be indexed by the distance between the apparatus101 and the object 113, which can be measured in real-time, and surfaceproperties of the object 113. Alternatively, the settings may becalculated in real-time using a suitable optimization algorithm findingat least a local minimum in respect of an expected proportion of theprojected electromagnetic radiation 111 to be captured which will bedirectly reflected electromagnetic radiation 121 by choosing arealizable time-variable distribution, aperture width and shutter scanspeed.

For illustrative purposes and to advance understanding of the presentdisclosure, FIG. 3 shows a fixed-spatial distribution 301 ofelectromagnetic radiation 111 that an example of the projector 109 canproduce and FIG. 8 shows a fixed-spatial distribution 801 ofelectromagnetic radiation 111 that another example of the projector 109can produce. It will be appreciated that the present disclosure is notlimited to the fixed-spatial distributions shown in FIGS. 3 and 8. Forcontinued illustrative purposes FIGS. 4A-C show example settingsaccording to which the method 201 may be performed with thefixed-spatial distribution 301 of FIG. 3. FIGS. 5A-C, 6A-C, and 7A-Cshow adjustments to those settings and the result of said adjustments.Likewise, FIGS. 9A-C show example settings according to which the method201 may be performed with the fixed-spatial distribution 801 of FIG. 8.FIGS. 10A-C, 11A-C, and 12A-C show adjustments to those settings and theresult of said adjustments.

The fixed-spatial distribution 301 of the projected electromagneticradiation 111, as shown in FIG. 3, comprises a repeating pattern. Inthis example the repeating pattern is in the form of a grid of dots.

FIG. 4A shows the aperture position in the y-axis with respect to timein the x-axis.

Between times t0 and t1 the FOV 107 of the rolling shutter from theaperture position is aligned with a strip of the surface 115 onto which,if projected, the first row of dots in the fixed-spatial distribution301 would be incident. There is no directly reflected electromagneticradiation 121 in the FOV 107 of the rolling shutter from the apertureposition between times t1 and t2. The FOV 107 in this aperture positionis aligned with a blank inter-row strip in the fixed spatialdistribution 301. In the example of FIG. 4A, the shutter scan speed andthe aperture width are constant during the image capture.

FIG. 4B shows the time-variable distribution 401 in a manner whichillustrates whether the electromagnetic radiation 111 is projected ornot (ON or OFF) in the y-axis with respect to time in the x-axis. Theelectromagnetic radiation 111 is not projected between times t0 and t1and is projected between times t1 and t2.

FIG. 4C shows the resulting amount of captured electromagnetic radiation111 which is directly reflected from the surface 115 of the object 113.Since the electromagnetic radiation 111 is not projected at times whenthe FOV 107 of the rolling shutter is aligned with the regions (strips)of the surface 115 where the electromagnetic radiation 111 would beincident, no directly reflected electromagnetic radiation 121 iscaptured. The electromagnetic radiation 111 is projected at times whenthe FOV 107 of the rolling shutter is aligned with a region (strip)adjacent that which is illuminated, and thus electromagnetic radiation123 which has undergone subsurface scattering is captured at each ofthese times.

In a general sense, it will be appreciated that the aperture width ofthe rolling shutter, the shutter scan speed of the rolling shutter andthe time-variable distribution 401 are controlled to cause capture ofthe first strips of the object 113 when second strips, adjacent thefirst, are illuminated with denser regions of the repeating pattern andto cause capture of the second strips when the projection of theelectromagnetic radiation 111 is turned off.

Consequently, in some examples, there is provided an apparatus 101comprising means for: capturing an image 209 of the object 113 using arolling shutter; during the image capture, projecting electromagneticradiation 111 with a fixed-spatial distribution 301 comprising arepeating pattern onto the object 113; and controlling timing of therolling shutter and projection of the electromagnetic radiation 111 tocause capture of first strips of the object 113 when second strips,adjacent the first, are illuminated by denser regions of the repeatingpattern, and to cause capture of the second strips while the projectionof the electromagnetic radiation 111 is turned off.

The effect is that the amount of projected electromagnetic radiation 111which is incident on an area adjacent an imaged area (strip) of theobject 113 at a given time is maximized while the amount of projectedelectromagnetic radiation 111 which is incident on the imaged area(strip) is minimized.

It should be noted that the aforementioned first strips are spaced fromeach other and the second strips are spaced from each other. The firstand second strips alternate to form the full imaged area of the object113. The strips are not necessarily of equal width. Each strip spansfully one dimension of the imaged area of the object 113.

FIGS. 5A-C show an example in which the aperture width is adjusted. Inthis case it is doubled with respect to the aperture width used in theexample of FIGS. 4A-C. The shutter scan speed and the time-variabledistribution 401 are not adjusted with respect to example of FIGS. 4A-C.The increased aperture width means that between, for example, times t1and t2 the FOV 107 of the rolling shutter now encompasses a region(strip) of the surface 115 onto which the first row of the fixed-spatialdistribution is incident. Accordingly, as shown in FIG. 5C, directlyreflected electromagnetic radiation 121 is captured. Thus, with theadjustment of the aperture width, a proportion of electromagneticradiation 111 captured which is directly reflected from the surface 115of the object 113 is increased.

FIGS. 6A-C show an example in which the shutter scan speed is adjusted.In this case it is doubled with respect to the shutter scan speed usedin the example of FIGS. 4A-C. The aperture width and the time-variabledistribution 401 are not adjusted with respect to example of FIGS. 4A-C.The increased shutter scan speed means that between, for example, timest1 and t2, the FOV 107 is aligned with the region (strip) of the surface115 onto which the first row of the fixed-spatial distribution 301 isincident for the first half of this time period. Accordingly, as shownin FIG. 6C, directly reflected electromagnetic radiation 121 iscaptured. Thus, with the adjustment of the shutter scan speed, aproportion of electromagnetic radiation 111 captured which is directlyreflected from the surface 115 of the object 113 is increased.

FIGS. 7A-C show an example in which the time-variable distribution isadjusted. The adjusted time-variable distribution is referenced in thisfigure as 701. In this case, the adjustment to the time-variabledistribution comprises a doubling of the duration of the off-cycle ascompared to the time-variable distribution 401 used in the example ofFIGS. 4A-C. The aperture width and the shutter scan speed are notadjusted with respect to example of FIGS. 4A-C. The increased durationof the off-cycle in the adjusted time-variable distribution 701 meansthat between, for example, times t2 and t3, the FOV 107 is aligned withthe region (strip) of the surface 115 onto which the second row of thefixed-spatial distribution 301 is incident. Accordingly, as shown inFIG. 7C, directly reflected electromagnetic radiation 121 is captured.Thus, with the adjustment of the time-variable distribution with whichthe electromagnetic radiation 111 is projected, a proportion ofelectromagnetic radiation 111 captured which is directly reflected fromthe surface 115 of the object 113 is increased.

The fixed spatial distribution 801 of the projected electromagneticradiation 111, as shown in FIG. 8, comprises a random distribution. Inthis example the random distribution is in the form of randomlydistributed dots.

FIG. 9A shows the aperture position in the y-axis with respect to timein the x-axis.

Between times t0 and t1 the FOV 107 of the rolling shutter from theaperture position is aligned with a strip of the surface 115 onto which,if projected, the first three dots by distance from the top of thefixed-spatial distribution 801 would be incident. Between times t1 andt2 the FOV 107 of the rolling shutter from the aperture position isaligned with a strip of the surface 115 onto which, if projected, thenext seven dots by distance from the top of the fixed-spatialdistribution 801 would be incident.

In the example of FIG. 9A, the shutter scan speed and the aperture widthare constant during the image capture.

FIG. 9B shows the time-variable distribution 901 in a manner whichillustrates whether the electromagnetic radiation 111 is projected ornot (ON or OFF) in the y-axis with respect to time in the x-axis. Theelectromagnetic radiation 111 is projected between times t0 and t1 andis not projected between times t1 and t2.

FIG. 9C shows the resulting amount of captured electromagnetic radiation111 which is directly reflected from the surface 115 of the object 113.

FIGS. 10A-C show an example in which the aperture width is adjusted. Inthis case it is halved with respect to the aperture width used in theexample of FIGS. 9A-C. The shutter scan speed and the time-variabledistribution 401 are not adjusted with respect to example of FIGS. 9A-C.The reduced aperture width in this instance does not affect which dotsin the fixed-spatial distribution 801 are captured as directly reflectedelectromagnetic radiation 121. Thus, with the adjustment of the aperturewidth, a proportion of electromagnetic radiation 111 captured which isdirectly reflected from the surface 115 of the object 113 is notdecreased.

FIGS. 11A-C show an example in which the shutter scan speed is adjusted.In this case it is doubled with respect to the shutter scan speed usedin the example of FIGS. 9A-C. The aperture width and the time-variabledistribution 801 are not adjusted with respect to example of FIGS. 9A-C.The increased shutter scan speed means that between, for example, timest0 and t1, the FOV 107 is successively aligned with the region (strip)of the surface 115 onto which the first three dots and then the nextseven dots of the fixed-spatial distribution 801 are incident.Accordingly, as shown in FIG. 110, more directly reflectedelectromagnetic radiation 121 is captured. Thus, with the adjustment ofthe shutter scan speed, a proportion of electromagnetic radiation 111captured which is directly reflected from the surface 115 of the object113 is increased.

FIGS. 12A-C show an example in which the time-variable distribution isadjusted. The adjusted time-variable distribution is referenced in thisfigure as 1201. In this case, the adjustment to the time-variabledistribution comprises a doubling of the strobing frequency andconsequentially a halving of the duration of the on- and off-cycles ascompared to the time-variable distribution 901 used in the example ofFIGS. 9A-C. The aperture width and the shutter scan speed are notadjusted with respect to example of FIGS. 9A-C. The increased strobingfrequency in the adjusted time-variable distribution 1201 means that theFOV 107 over the course of the image capture encompasses the regionsilluminated by each dot in the fixed-spatial distribution 801 while theyare illuminated. Accordingly, as shown in FIG. 12C, more directlyreflected electromagnetic radiation 121 is captured. Thus, with theadjustment of the time-variable distribution with which theelectromagnetic radiation 111 is projected, a proportion ofelectromagnetic radiation 111 captured which is directly reflected fromthe surface 115 of the object 113 is increased.

FIG. 13 shows an example of a method 1301 in which the apparatus 101 isused to capture multiple images 209.

Blocks 203, 205, and 207 of method 201 are repeated in the method 1301to obtain multiple images 209. Block 1303 follows block 207, the end ofa single image capture when a strip of the object 113 is captured at thefinal aperture position.

At block 1303 of the method 1301 the images 209 are recorded. Therecording of images 209 may comprise only temporary recording, or it maycomprise permanent recording or it may comprise both temporary recordingand permanent recording. Temporary recording implies the recording ofdata temporarily. This may, for example, occur during sensing or imagecapture, occur at a dynamic memory, occur at a buffer such as a circularbuffer, a register, a cache or similar. Permanent recording implies thatthe data is in the form of an addressable data structure that isretrievable from an addressable memory space and can therefore be storedand retrieved until deleted or over-written, although long-term storagemay or may not occur. The use of the term “capture” in relation to animage relates to temporary recording of the data of the image. The useof the term “store” in relation to an image relates to permanentrecording of the data of the image.

At block 1305 of the method 1301, which also follows block 207, acapture-related parameter is varied before the method 201 is repeated.Accordingly, for different image captures, different capture-relatedparameters are used.

Examples of capture-related parameters that may be varied at block 1305include, without limitation: a relative position of the apparatus 101and object 113; a frequency and/or amplitude of the projectedelectromagnetic radiation 111; the time-variable distribution of theprojected electromagnetic radiation 111; the aperture width of therolling shutter; and/or the shutter scan speed of the rolling shutter.

Multiple images 209 of the object 113 can be captured and, from oneimage capture to another, a relative position of the apparatus 101 andobject 113 is varied. This enables information about the full subsurfacestructure 119 to be obtained where the fixed-spatial distribution is toosparse (for example, the distance between adjacent dots is too great) toenable the capture of electromagnetic radiation 123 which has undergonesubsurface scattering from certain parts of the subsurface structure 119unless the fixed-spatial distribution is moved relative to the object113. Examples of such sparse distributions may include, for example,those produced by rear facing LI DAR devices which are becomingintegrated into smartphones.

Multiple images 209 of the object 113 can be captured and, from oneimage capture to another, a frequency and/or amplitude of the projectedelectromagnetic radiation 111 is varied. For different image captures,different frequencies and/or amplitudes of the projected electromagneticradiation 111 are used. Subsurface scattering effects for certainsubsurface structures 119 may be better realised with differentfrequencies or amplitudes of projected electromagnetic radiation 111.

Multiple images 209 of the object 113 can be captured and, from oneimage capture to another, the time-variable distribution of theprojected electromagnetic radiation 111 is varied. For different imagecaptures, different time-variable distributions of the projectedelectromagnetic radiation 111 are used.

Multiple images 209 of the object 113 can be captured and, from oneimage capture to another, the aperture width and/or the shutter scanspeed of the rolling shutter is varied. For different image captures,different aperture widths and/or the shutter scan speeds of the rollingshutter are used.

At block 1307 of the method 1301 two or more images 209 are fused into acombined image 1309 of the object 113.

The combined image 1309 may comprise less directly reflectedelectromagnetic radiation 121 than either of the two or more images 209.The two or more images 209 may be selected from amongst those recordedin order that, when fused, the resultant combined image 1309 comprisesless directly reflected electromagnetic radiation 121 than either of thetwo or more images 209. The manner in which two or more images 209 arefused may be configured to ensure that the resultant combined image 1309comprises less directly reflected electromagnetic radiation 121 thaneither of the two or more images 209.

FIG. 14A shows an example of the apparatus 101 being moved in respect ofthe object 113 such that a relative position r of the apparatus 101 andthe object 113 varies. The apparatus 101 repeats the method 201 whilethe relative position r of the apparatus 101 and the object 113 variesin order to capture multiple spatially-offset images 209 of the object113.

The relative position r of the apparatus 101 and the object 113 may varyas a result of jitter. Jitter refers to motion that has high temporalfrequency relative to the total time for the multiple image capturesequence. The variations in relative position r of the apparatus 101 andthe object 113 due to jitter may be low magnitude, comparable to FOV 107of the rolling shutter at a given aperture position. Jitter, asillustrated in FIG. 14B, can result from vibration in the apparatus 101or camera 103, which can be deliberately induced by operation ofactuators comprised in the apparatus 101, or can result from theunstable hand of a user of the apparatus 101. Accordingly, the presentdisclosure can opportunistically take advantage of the inherentlyunstable manner in which the user controls the relative position r ofthe apparatus 101 and the object 113 by capturing the multiplespatially-offset images 209 during jitter.

The effect of a variation in the relative position r of the apparatus101 and the object 113 on the distribution 1501 of illumination, by theprojected electromagnetic radiation 111, of the surface 115 of theobject 113 is shown in FIGS. 15A and 15B where the distribution beforethe variation is referenced as 1501 and the distribution after thevariation is referenced as 1501′.

As per block 1307 of the method 1301, two or more spatially-offsetimages can be fused into a combined image 1309 of the object whichcomprises less directly reflected electromagnetic radiation 121 thaneither of the two or more spatially-offset images.

Consequently, in some examples, there is provided an apparatus 101comprising means for: capturing multiple spatially-offset images 209 ofthe object 113 as a relative position r of the apparatus 101 and object113 varies due to jitter; during the image captures, projectingelectromagnetic radiation 111 onto the object 113; and fusing two ormore of the spatially-offset images 209 into a combined image 1309 ofthe object 113 which comprises less of the projected electromagneticradiation 111 directly reflected from the surface 115 of the object 113than either of the two or more spatially-offset images 209.

In some examples, subsurface structure information obtained from themultiple spatially-offset images 209 can be used to align the multiplespatially-offset images 209. Continuity of subsurface structures 119 canbe expected. Therefore, even though differently located fragments of thesubsurface structure 119 may be inferred from different ones of themultiple spatially-offset images 209, patterns can be predicted and thefragments can be mapped to the patterns to enable alignment.

FIG. 16 shows an example of a method 1601 of determining a targetfrequency and/or target amplitude for the projected electromagneticradiation 111.

Block 1603 of the method 1601 comprises obtaining surface properties1605 of the object 113.

Surface properties 1605 can include, for example, reflectance andabsorption. Surface properties can be can be detected using a testillumination of the surface 115 or could be looked up based on knowledgeof the object 113 being or to be imaged. Knowledge of the object 113being or to be imaged can be obtained by, for example, computer vision.

At block 1617 of the method 1601 the target frequency 1619 and/or targetamplitude 1621 for the projected electromagnetic radiation 111 isdetermined based on the surface properties 1605 and a target depth 1607of scattering and/or target lateral distance 1609 of scattering. Thetarget depth 1607 can be based on the type of object 113 to be imaged.The target lateral distance 1609 of scattering can be based on adistribution of illumination 1501 of the object 113 by the projectedelectromagnetic radiation 111.

Determining the target frequency 1619 and/or amplitude 1621 for theprojected electromagnetic radiation 111 can be further based on adistance 1611 between the apparatus 101 and the object 113 and/or onambient lighting conditions 1613. Guidance can be provided to a userwhich indicates a distance 1611 between the apparatus 101 and the object113 to facilitate the target depth 1607 of scattering and/or targetlateral distance 1609 of scattering.

Determining the target frequency 1619 and/or amplitude 1621 for theprojected electromagnetic radiation 111 can be further based onsubsurface scattering properties 1615 of a material from which theobject 113 is composed. Subsurface scattering properties 1615 may beestimated from adjacent pixel values to the direct illuminationlocations.

FIG. 17 shows an example of a filtering method 1701 which processes theimage 209 of the object 113 or the combined image 1309 of the object 113to filter directly reflected electromagnetic radiation 121, in the eventthat any directly reflected electromagnetic radiation 121 is captured,from electromagnetic radiation 123 which has undergone lateralsubsurface scattering based on the distribution 1501 of illumination ofthe object 113.

Block 1703 of the filtering method 1701 comprises determining thedistribution 1501 of illumination of the object 113 by the projectedelectromagnetic radiation 111.

This can be achieved by acquiring an image of the object 113 asilluminated by the projected electromagnetic radiation 111, andprocessing the image to detect the distribution of illumination. In analternative example, the distribution of illumination can be predictedgiven the known fixed-spatial distribution 301, 801 of the projectedelectromagnetic radiation 111 and the distance 1611 to the surface 115of the object 113. The prediction may also take into account thegeometry of the object's surface 115, with the distribution ofillumination on the object's surface 115 being geometrically distorted,in comparison to the fixed-spatial distribution 301, 801 with which theelectromagnetic radiation 111 was projected, due to the geometry of thesurface 115.

At block 1705 of the method 1701 a filter 1707 is obtained based on thedistribution 1501 of illumination.

At block 1709 of the method 1701 the filter 1707 is applied to the image209 or the combined image 1309 of the object 113. The output of theapplication of the filter 1707 is an image 1711 composed from capturedelectromagnetic radiation 123 which has undergone subsurface scattering.Analysis can then be performed to determine subsurface structureinformation.

Knowledge of the distribution of illumination 1501 and capture-relatedparameters, such as the amplitude and frequency of the projectedelectromagnetic radiation 111, can be used in the analysis to determinethe direction and distance travelled within the subsurface region 117 byelectromagnetic radiation 123 which has undergone subsurface scatteringinto the FOV 107 at any given aperture position. The distance anddirection of travel in the subsurface region 117 can be used to infersubsurface structure 119. For example, shadows cast by subsurfacestructures 119 can be detected from the decreased re-emission from thesurface 115 at locations downstream of the location at which theprojected electromagnetic radiation 111 is incident. Knowledge of thelocation at which the projected electromagnetic radiation 111 isincident enables the use of shadow information for resolving thelocation of subsurface structures 119. Furthermore, the penetrationdepth of the electromagnetic radiation 111 undergoing subsurfacescattering may reduce as it moves laterally away from the location atwhich it was incident on the surface 115. The lateral distance betweenthe detected location of the subsurface structure 119 and the locationat which the electromagnetic radiation 111 was incident can therefore beused to give a range or estimate of the depth of at which the subsurfacestructure 119 is located.

The above described examples find application as enabling components of,for example, user authentication systems. For example, the object 113imaged can be a body part of a user. Where the object 113 is a body partof a user, a map of the user's subsurface blood vessels (such as one ormore of: veins, arteries, capillaries, etc.) can be obtained from theimage 209 of the object 113 or the combined image 1309 of the object113. The user can be authenticated based on the obtained map.

Access to at least one application can be controlled in dependence onwhether or not the user is an authorised user. The at least oneapplication may be a function or set of functions that the apparatus 101is configured to perform or enable. Blood vessel structures of anauthorised user can be recorded in the form of an addressable datastructure that is retrievable from an addressable memory space and cantherefore be stored and retrieved until deleted or over-written,although long-term storage may or may not occur. The recorded bloodvessel structures of an authorised user can be retrieved for comparisonwith the obtained map of the user's subsurface blood vessels. Thecomparison may be made using correlations, pattern recognition or anyother suitable process. If the comparison shows a good match orcorrelation then access to at least one application can be permitted. Ifthe comparison does not show a good match or correlation then access toat least one application can be denied.

The image capture of a body part of the user can be caused in responseto a trigger event. The trigger event may be, for example, a user inputto the apparatus 101, a predefined routine, or a request from one ormore applications run on the apparatus 101. The request from the one ormore applications may be made when authentication of the user isrequired in respect of access to the one or more applications.

The above described examples can also find application as enablingcomponents of, for example, material analysis systems such as productauthentication purpose including clothing material identification, foodanalysis or others. Subsurface analysis of food can reveal ripeness orspoilage. Subsurface analysis of skin can be used in health monitoringand disease diagnosis.

Although in the foregoing the settings of camera- and projector-relatedparameters have been described as minimising the proportion of capturedelectromagnetic radiation 111 which is directly reflectedelectromagnetic radiation 121 in the image 209, it is to be appreciatedthat only some parts of the object 113 may actually be of interest forthe application in which the above described examples are used. In suchinstances, if the proportion of projected electromagnetic radiation 111captured which is directly reflected from one or more surfaces 115 ofone or more parts of the object 113 which are of interest can beminimised, then it will not matter if the overall proportion of capturedelectromagnetic radiation 111 which is directly reflectedelectromagnetic radiation 121 in the image 209 is not minimised.

Accordingly, in some examples the time-variable distribution of theprojected electromagnetic radiation 111, the aperture width of therolling shutter, and the shutter scan speed of the rolling shutter, aresuch that adjustment of one or more of these would not decrease theproportion of projected electromagnetic radiation 111 captured which isdirectly reflected from one or more surfaces 115 of one or more parts ofthe object 113 which are of interest.

Parts of the object 113 which are of interest can comprise those partshaving subsurface structures 119 of interest. For example, parts of auser's face comprising differential blood vessel structures betweendifferent people may be considered parts of the face which are ofinterest. Examples of parts of a user's face that may be of interestinclude, without limitation: the skin above the top lip which holds thesuperior labial artery; the skin below the bottom lip which holds theinferior labial artery; and the areas to the left and right of the nosewhich hold the facial arteries.

Although in the foregoing the settings of camera- and projector-relatedparameters and of other capture-related parameters such as frequencyand/or amplitude of the projected electromagnetic radiation 111 areconsistent throughout a single image capture according to method 201, itis to be appreciated that subsurface scattering effects for certainsubsurface structures 119 may be better realised with differentsettings. While one option is to capture multiple images 209 wheresettings are varied from one image capture to another, as described inrelation to FIG. 13, another option is to vary the settings during theimage capture.

For example, the shutter scan speed of the rolling shutter need not beconstant during the image capture and, likewise, the aperture width ofthe rolling shutter need not be constant during the image capture.

In some examples the target frequency 1619 and/or amplitude 1621 can bedifferentially determined in respect of different parts of the object113. The frequency and/or amplitude of the projected electromagneticradiation 111 during the image capture can be varied such that differentparts of the object 113 are illuminated with electromagnetic radiation111 projected at respective target frequencies 1619 and/or amplitudes1621.

FIG. 18 illustrates an example of a controller 1801. Implementation ofthe controller 1801 may be as controller circuitry. The controller 1801may be implemented in hardware alone, have certain aspects in softwareincluding firmware alone or can be a combination of hardware andsoftware (including firmware).

As illustrated in FIG. 18 the controller 1801 may be implemented usinginstructions that enable hardware functionality, for example, by usingexecutable instructions of a computer program 1807 in a general-purposeor special-purpose processor 1803 that may be stored on a computerreadable storage medium (disk, memory, etc.) to be executed by such aprocessor 1803.

The processor 1803 is configured to read from and write to the memory1805. The processor 1803 may also comprise an output interface via whichdata and/or commands are output by the processor 1803 and an inputinterface via which data and/or commands are input to the processor1803.

The memory 1805 stores a computer program 1807 comprising computerprogram instructions (computer program code 1809) that controls theoperation of the apparatus 101 when loaded into the processor 1803. Thecomputer program instructions, of the computer program 1807, provide thelogic and routines that enables the apparatus 101 to perform the method201 illustrated in FIG. 2 and optionally the methods illustrated inFIGS. 13, 16, and 17. The processor 1803 by reading the memory 1805 isable to load and execute the computer program 1807.

The apparatus 101 therefore comprises: at least one processor 1803; andat least one memory 1805 including computer program code 1809, the atleast one memory 1805 and the computer program code 1809 configured to,with the at least one processor 1803, cause the apparatus 101 at leastto perform: capturing 203 an image 209 of an object 113 using a rollingshutter having an aperture width and shutter scan speed; during theimage capture, projecting 205 electromagnetic radiation 111 with afixed-spatial 301, 801, time-variable 401, 901 distribution onto theobject 113, wherein the time-variable distribution 401, 901 of theprojected electromagnetic radiation 111, the aperture width of therolling shutter, and the shutter scan speed of the rolling shutter, aresuch that adjustment of one or more of these would not decrease aproportion of projected electromagnetic radiation 111 captured which isdirectly reflected from a surface 115 of the object 113.

As illustrated in FIG. 18, the computer program 1807 may arrive at theapparatus 101 via any suitable delivery mechanism 1811. The deliverymechanism 1811 may be, for example, a machine readable medium, acomputer-readable medium, a non-transitory computer-readable storagemedium, a computer program product, a memory device, a record mediumsuch as a Compact Disc Read-Only Memory (CD-ROM) or a Digital VersatileDisc (DVD) or a solid state memory, an article of manufacture thatcomprises or tangibly embodies the computer program 1807. The deliverymechanism may be a signal configured to reliably transfer the computerprogram 1807. The apparatus 101 may propagate or transmit the computerprogram 1807 as a computer data signal. In some examples the computerprogram 1807 may be transmitted to the apparatus 101 using a wirelessprotocol such as Bluetooth, Bluetooth Low Energy, Bluetooth Smart,6LoWPan (IPv6 over low power personal area networks) ZigBee, ANT+, nearfield communication (NFC), Radio frequency identification, wirelesslocal area network (wireless LAN) or any other suitable protocol.

In some examples there is provided computer program instructions 1809for causing an apparatus 101 to perform at least the following or forperforming at least the following: causing capture 203 of an image 209of an object 113 using a rolling shutter having an aperture width andshutter scan speed; during the image capture, causing projection 205 ofelectromagnetic radiation 111 with a fixed-spatial 301, 801,time-variable 401, 901 distribution onto the object 113, wherein thetime-variable distribution 401, 901 of the projected electromagneticradiation 111, the aperture width of the rolling shutter, and theshutter scan speed of the rolling shutter, are such that adjustment ofone or more of these would not decrease a proportion of projectedelectromagnetic radiation 111 captured which is directly reflected froma surface 115 of the object 113.

The computer program instructions 1809 may be comprised in a computerprogram 1807, a non-transitory computer readable medium, a computerprogram product, a machine readable medium. In some but not necessarilyall examples, the computer program instructions 1809 may be distributedover more than one computer program 1807.

Although the memory 1805 is illustrated as a single component/circuitryit may be implemented as one or more separate components/circuitry someor all of which may be integrated/removable and/or may providepermanent/semi-permanent/dynamic/cached storage.

Although the processor 1803 is illustrated as a singlecomponent/circuitry it may be implemented as one or more separatecomponents/circuitry some or all of which may be integrated/removable.The processor 1803 may be a single core or multi-core processor.

References to “computer-readable storage medium”, “computer programproduct”, “tangibly embodied computer program” etc. or a “controller”,“computer”, “processor” etc. should be understood to encompass not onlycomputers having different architectures such as single/multi-processorarchitectures and sequential (Von Neumann)/parallel architectures butalso specialized circuits such as field-programmable gate arrays (FPGA),application specific circuits (ASIC), signal processing devices andother processing circuitry. References to computer program,instructions, code etc. should be understood to encompass software for aprogrammable processor or firmware such as, for example, theprogrammable content of a hardware device whether instructions for aprocessor, or configuration settings for a fixed-function device, gatearray or programmable logic device etc.

As used in this application, the term “circuitry” may refer to one ormore or all of the following:

(a) hardware-only circuitry implementations (such as implementations inonly analog and/or digital circuitry) and

(b) combinations of hardware circuits and software, such as (asapplicable):

(i) a combination of analog and/or digital hardware circuit(s) withsoftware/firmware and

(ii) any portions of hardware processor(s) with software (includingdigital signal processor(s)), software, and memory(ies) that worktogether to cause an apparatus, such as a mobile phone or server, toperform various functions and

(c) hardware circuit(s) and or processor(s), such as a microprocessor(s)or a portion of a microprocessor(s), that requires software (e.g.firmware) for operation, but the software may not be present when it isnot needed for operation.

This definition of circuitry applies to all uses of this term in thisapplication, including in any claims. As a further example, as used inthis application, the term circuitry also covers an implementation ofmerely a hardware circuit or processor and its (or their) accompanyingsoftware and/or firmware. The term circuitry also covers, for exampleand if applicable to the particular claim element, a baseband integratedcircuit for a mobile device or a similar integrated circuit in a server,a cellular network device, or other computing or network device.

The blocks illustrated in FIG. 2 and optionally those illustrated inFIGS. 13, 16, and 17 may represent steps in a method and/or sections ofcode in the computer program 1807. The illustration of a particularorder to the blocks does not necessarily imply that there is a requiredor preferred order for the blocks and the order and arrangement of theblock may be varied. Furthermore, it may be possible for some blocks tobe omitted.

Where a structural feature has been described, it may be replaced bymeans for performing one or more of the functions of the structuralfeature whether that function or those functions are explicitly orimplicitly described.

Consequently, in some examples, the apparatus 201 comprises means for:capturing 203 an image 209 of an object 113 using a rolling shutterhaving an aperture width and shutter scan speed; during the imagecapture, projecting 205 electromagnetic radiation 111 with afixed-spatial 301, 801, time-variable 401, 901 distribution onto theobject 113, wherein the time-variable distribution 401, 901 of theprojected electromagnetic radiation 111, the aperture width of therolling shutter, and the shutter scan speed of the rolling shutter, aresuch that adjustment of one or more of these would not decrease aproportion of projected electromagnetic radiation 111 captured which isdirectly reflected from a surface 115 of the object 113.

In some but not necessarily all examples, the apparatus 101 isconfigured to communicate data from the apparatus 101 with or withoutlocal storage of the data in a memory 1805 at the apparatus 101 and withor without local processing of the data by circuitry or processors 1803at the apparatus 101.

The data may, for example, be the image 209, combined image 1309 orinformation about the subsurface structure 119 produced by theprocessing of the image 209 or combined image 1309.

The data may be stored in processed or unprocessed format remotely atone or more devices. The data may be stored in the Cloud.

The data may be processed remotely at one or more devices. The data maybe partially processed locally and partially processed remotely at oneor more devices.

The data may be communicated to the remote devices wirelessly via shortrange radio communications such as Wi-Fi or Bluetooth, for example, orover long-range cellular radio links. The apparatus may comprise acommunications interface such as, for example, a radio transceiver forcommunication of data.

The apparatus 101 may be part of the Internet of Things forming part ofa larger, distributed network.

The processing of the data, whether local or remote, may be for thepurpose of health monitoring, data aggregation, patient monitoring,vital signs monitoring or other purposes.

The processing of the data, whether local or remote, may involveartificial intelligence or machine learning algorithms. The data may,for example, be used as learning input to train a machine learningnetwork or may be used as a query input to a machine learning network,which provides a response. The machine learning network may for exampleuse linear regression, logistic regression, vector support machines oran acyclic machine learning network such as a single or multi hiddenlayer neural network.

The processing of the data, whether local or remote, may produce anoutput. The output may be communicated to the apparatus 101 where it mayproduce an output sensible to the user such as an audio output, visualoutput or haptic output.

Machine learning, which can include statistical learning, is a field ofcomputer science that gives computers the ability to learn without beingexplicitly programmed. The computer learns from experience E withrespect to some class of tasks T and performance measure P if itsperformance at tasks in T, as measured by P, improves with experience E.The computer can often learn from prior training data to makepredictions on future data. Machine learning includes wholly orpartially supervised learning and wholly or partially unsupervisedlearning. It may enable discrete outputs (for example classification,clustering) and continuous outputs (for example regression). Machinelearning may for example be implemented using different approaches suchas cost function minimization, artificial neural networks, supportvector machines and Bayesian networks for example. Cost functionminimization may, for example, be used in linear and polynomialregression and K-means clustering. Artificial neural networks, forexample with one or more hidden layers, model complex relationshipbetween input vectors and output vectors. Support vector machines may beused for supervised learning. A Bayesian network is a directed acyclicgraph that represents the conditional independence of a number of randomvariables.

The term “comprise” is used in this document with an inclusive not anexclusive meaning. That is any reference to X comprising Y indicatesthat X may comprise only one Y or may comprise more than one Y. If it isintended to use “comprise” with an exclusive meaning then it will bemade clear in the context by referring to “comprising only one” or byusing “consisting”.

In this description, reference has been made to various examples. Thedescription of features or functions in relation to an example indicatesthat those features or functions are present in that example. The use ofthe term “example” or “for example” or “can” or “may” in the textdenotes, whether explicitly stated or not, that such features orfunctions are present in at least the described example, whetherdescribed as an example or not, and that they can be, but are notnecessarily, present in some of or all other examples. Thus “example”,“for example”, “can” or “may” refers to a particular instance in a classof examples. A property of the instance can be a property of only thatinstance or a property of the class or a property of a sub-class of theclass that includes some but not all of the instances in the class. Itis therefore implicitly disclosed that a feature described withreference to one example but not with reference to another example, canwhere possible be used in that other example as part of a workingcombination but does not necessarily have to be used in that otherexample.

Although examples have been described in the preceding paragraphs withreference to various examples, it should be appreciated thatmodifications to the examples given can be made without departing fromthe scope of the claims.

Features described in the preceding description may be used incombinations other than the combinations explicitly described above.

Although functions have been described with reference to certainfeatures, those functions may be performable by other features whetherdescribed or not.

Although features have been described with reference to certainexamples, those features may also be present in other examples whetherdescribed or not.

The term “a” or “the” is used in this document with an inclusive not anexclusive meaning. That is any reference to X comprising a/the Yindicates that X may comprise only one Y or may comprise more than one Yunless the context clearly indicates the contrary. If it is intended touse “a” or “the” with an exclusive meaning then it will be made clear inthe context. In some circumstances the use of ‘at least one’ or ‘one ormore’ may be used to emphasis an inclusive meaning but the absence ofthese terms should not be taken to infer any exclusive meaning.

The presence of a feature (or combination of features) in a claim is areference to that feature or (combination of features) itself and alsoto features that achieve substantially the same technical effect(equivalent features). The equivalent features include, for example,features that are variants and achieve substantially the same result insubstantially the same way. The equivalent features include, forexample, features that perform substantially the same function, insubstantially the same way to achieve substantially the same result.

In this description, reference has been made to various examples usingadjectives or adjectival phrases to describe characteristics of theexamples. Such a description of a characteristic in relation to anexample indicates that the characteristic is present in some examplesexactly as described and is present in other examples substantially asdescribed.

Whilst endeavoring in the foregoing specification to draw attention tothose features believed to be of importance it should be understood thatthe Applicant may seek protection via the claims in respect of anypatentable feature or combination of features hereinbefore referred toand/or shown in the drawings whether or not emphasis has been placedthereon.

We claim:
 1. An apparatus comprising: at least one processor; and atleast one memory including computer program code, the at least onememory and the computer program code configured to, with the at leastone processor, cause the apparatus at least to: capture an image of anobject using a rolling shutter having an aperture width and shutter scanspeed; during the image capture, project electromagnetic radiation witha fixed-spatial, time-variable distribution onto the object, wherein thetime-variable distribution of the projected electromagnetic radiation,the aperture width of the rolling shutter, and the shutter scan speed ofthe rolling shutter, are such that adjustment of one or more of thesewould not decrease a proportion of projected electromagnetic radiationcaptured which is directly reflected from a surface of the object;wherein the fixed-spatial distribution of the projected electromagneticradiation comprises a repeating pattern, and wherein the at least onememory and the computer program code are configured to, with the atleast one processor, further cause the apparatus to: control thetime-variable distribution of the projected electromagnetic radiation,the aperture width of the rolling shutter, and the shutter scan speed ofthe rolling shutter to: cause capture of first strips of the object whensecond strips, adjacent the first, are illuminated by denser regions ofthe repeating pattern; and cause capture of the second strips while theprojection of the electromagnetic radiation is turned off.
 2. Theapparatus of claim 1 wherein the at least one memory and the computerprogram code are configured to, with the at least one processor, furthercause the apparatus to: capture, at a given aperture position of therolling shutter, electromagnetic radiation which is projected onto anarea of the object outside a field of view from the aperture positionand which has undergone subsurface scattering into the field of view. 3.The apparatus of claim 1 wherein the at least one memory and thecomputer program code are configured to, with the at least oneprocessor, further cause the apparatus to: set the time-variabledistribution of the projected electromagnetic radiation, the aperturewidth of the rolling shutter, and the shutter scan speed of the rollingshutter, such that adjustment of one or more of these would not decreasethe proportion of projected electromagnetic radiation captured which isdirectly reflected from a surface of the object.
 4. The apparatus ofclaim 1 wherein the time-variable distribution and the shutter scanspeed are such that over periods of the time-variable distribution inwhich the electromagnetic radiation is not projected the change in theaperture position is less than or equal to twice a target lateraldistance for scattering.
 5. The apparatus of claim 1 wherein thetime-variable distribution of the projected electromagnetic radiation,the aperture width of the rolling shutter, and the shutter scan speed ofthe rolling shutter, are such that adjustment of one or more of thesewould not decrease the proportion of projected electromagnetic radiationcaptured which is directly reflected from one or more surfaces of one ormore parts of the object which are of interest.
 6. The apparatus ofclaim 1 wherein the at least one memory and the computer program codeare configured to, with the at least one processor, further cause theapparatus to: capture multiple spatially-offset images of the object asa relative position of the apparatus and the object varies.
 7. Theapparatus of claim 6 wherein the multiple spatially-offset images arecaptured during jitter.
 8. The apparatus of claim 6 wherein the at leastone memory and the computer program code are configured to, with the atleast one processor, further cause the apparatus to: fuse two or morespatially-offset images into a combined image of the object whichcomprises less of the projected electromagnetic radiation directlyreflected from a surface of the object than either of the two or morespatially-offset images.
 9. The apparatus of claim 1 wherein the atleast one memory and the computer program code are configured to, withthe at least one processor, further cause the apparatus to: capturemultiple images of the object and to vary, from one image capture toanother, one or more of: a frequency of the projected electromagneticradiation; an amplitude of the projected electromagnetic radiation; thetime-variable distribution of the projected electromagnetic radiation;the aperture width of the rolling shutter; or the shutter scan speed ofthe rolling shutter.
 10. The apparatus of claim 1 wherein the at leastone memory and the computer program code are configured to, with the atleast one processor, further cause the apparatus to: obtain surfaceproperties of the object; and, based on the surface properties and atarget depth of scattering or target lateral distance of scattering,determine a target frequency and/or amplitude for the projectedelectromagnetic radiation.
 11. The apparatus of claim 1 wherein the atleast one memory and the computer program code are configured to, withthe at least one processor, further cause the apparatus to: analyse theimage of the object or the combined image of the object to determinesubsurface structure information.
 12. The apparatus of claim 1 whereinthe object imaged is a body part of a user and the at least one memoryand the computer program code are configured to, with the at least oneprocessor, further cause the apparatus to: obtain, from the image of theobject or the combined image of the object, a map of the user'ssubsurface blood vessels; and authenticate the user based on theobtained map.
 13. A method comprising: capturing an image of an objectusing a rolling shutter having an aperture width and shutter scan speed;during the image capture, projecting electromagnetic radiation with afixed-spatial, time-variable distribution onto the object, wherein thetime-variable distribution of the projected electromagnetic radiation,the aperture width of the rolling shutter, and the shutter scan speed ofthe rolling shutter, are such that adjustment of one or more of thesewould not decrease a proportion of projected electromagnetic radiationcaptured which is directly reflected from a surface of the object;wherein the fixed-spatial distribution of the projected electromagneticradiation comprises a repeating pattern, and wherein the method furthercomprises: controlling the time-variable distribution of the projectedelectromagnetic radiation, the aperture width of the rolling shutter,and the shutter scan speed of the rolling shutter to: cause capture offirst strips of the object when second strips, adjacent the first, areilluminated by denser regions of the repeating pattern; and causecapture of the second strips while the projection of the electromagneticradiation is turned off.
 14. The method of claim 13, wherein the objectimaged is a body part of a user and wherein the method furthercomprises: obtaining, from the image of the object or the combined imageof the object, a map of the user's subsurface blood vessels; andauthenticating the user based on the obtained map.
 15. The method ofclaim 13 further comprising: analysing the image of the object or thecombined image of the object to determine subsurface structureinformation.
 16. A non-transitory computer readable medium comprisingprogram instructions stored thereon for performing at least thefollowing: causing capture of an image of an object using a rollingshutter having an aperture width and shutter scan speed; during theimage capture, causing projection of electromagnetic radiation with afixed-spatial, time-variable distribution onto the object, wherein thetime-variable distribution of the projected electromagnetic radiation,the aperture width of the rolling shutter, and the shutter scan speed ofthe rolling shutter, are such that adjustment of one or more of thesewould not decrease a proportion of projected electromagnetic radiationcaptured which is directly reflected from a surface of the object;wherein the program instructions are further configured to cause:controlling the time-variable distribution of the projectedelectromagnetic radiation, the aperture width of the rolling shutter,and the shutter scan speed of the rolling shutter to: cause capture offirst strips of the object when second strips, adjacent the first, areilluminated by denser regions of the repeating pattern; and causecapture of the second strips while the projection of the electromagneticradiation is turned off.
 17. The non-transitory computer readable mediumof claim 16, wherein the object imaged is a body part of a user, andwherein the program instructions are further configured to cause:obtaining, from the image of the object or the combined image of theobject, a map of the user's subsurface blood vessels; and authenticatingthe user based on the obtained map.